@ofborg build pkgsCross.aarch64-multiplatform.bootstrapTools.build

Nits:

• Our release.nix calls this stdenvBootstrapTools, so perhaps reuse the same name?
• This PR's code won't work on *-darwin. Maybe that would be confusing, as we actually do have the tools for those platforms as well? (and e.g. OfBorg does support the platforms)

Our release.nix calls this stdenvBootstrapTools, so perhaps reuse the same name?

Done.

This PR's code won't work on *-darwin. Maybe that would be confusing, as we actually do have the tools for those platforms as well? (and e.g. OfBorg does support the platforms)

Done. Thanks for reminding me. Since ofborg can build the darwin bootstrap-tools this PR should certainly accommodate that.

I looked into generalizing this further, but there really are only three bootstrap-files-builders: linux, darwin, and freebsd... and the freebsd builder doesn't appear to work anymore (I can't get pkgsCross to grok its system-double).

There wasn't an externally-visible way to invoke the "map this hostPlatform to the subdirectory of pkgs/stdenv that builds its boostrap-files" function in pkgs/stdenv/default.nix, and since there are only two working flavors right now I didn't think it was worthwhile to go refactoring pkgs/stdenv/default.nix in order to expose that.

I wouldn't worry about BSD here. IIRC and on a quick glance, the bootstrap we use for FreeBSD just impurely reuses /usr/bin/* tools: https://github.com/NixOS/nixpkgs/blob/487624c15195/pkgs/stdenv/freebsd/trivial-bootstrap.sh (and I haven't heard of anyone using NixPkgs on FreeBSD for a long time)

@ofborg build stdenvBootstrapTools
(I wonder if this picks up attribute sets/maps)

I like this. I just wonder about aarch64-darwin, as currently we're cross-compiling the bootstrap tools (as you can see in release.nix), but this attribute builds them natively.

This difference might be confusing. Still, maybe we'd better switch to the native approach here as well (soon-ish), in which case I think we could leave this PR simple as-is, and the difference would disappear. But darwin is not my topic; I guess /cc @NixOS/darwin-maintainers?

Ugh, yeah, I don't think that expression should be setting the system attribute... it leaves you with no way of saying "I want a native compile". Or of using those expressions on any other buildPlatform than the one that is hardcoded in the system attribute.

But I think that's a concern to be left for a separate PR.

This looks like a good idea. A next step could be making it possible to get a stdenv with a bootstrap-tools build.

This looks like a good idea. A next step could be making it possible to get a stdenv with a bootstrap-tools build.

Yes, one of my longer-term goals is to have an automated Hydra job ensuring that generating fresh bootstrap-files tarballs wouldn't break anything. I worry about bitrot there, like one day we wake up and realize that we can't rebuild the bootstrap-files tarballs without breaking stuff.

Actually regenerating the tarballs periodically (each nixos release?) probably isn't a good idea though. Since each regenerated tarball's provenance includes the previous generation this just creates a really really really long dependency chain for reproducibility, and you've still got a "magic tarball hash" at the start of that chain. So making sure we can regenerate the bootstrap-tarballs is a good idea, but actually doing that (unless we're forced to) probably isn't.

We had .test in these stdenvBootstrapTools, running on Hydra: https://hydra.nixos.org/eval/1757655?filter=stdenvboot (maybe I missed your point)

When I wrote

one day we wake up and realize that we can't rebuild the bootstrap-files tarballs without breaking stuff.

I should have said "without breaking the stdenv bootstrap process" instead of "stuff".

We had .test in these stdenvBootstrapTools, running on Hydra: https://hydra.nixos.org/eval/1757655?filter=stdenvboot (maybe I missed your point)

It looks like test-bootstrap-tools rebuilds the bootstrap tools, unpacks them, and uses them to (a) run gcc on a one-liner "hello world", and (b) check that foo --version doesn't crash for foo in {sed, find, diff, patch, make, awk, grep, gcc}. That's good, it'll let us know if the bootstrap-files become totally broken at some point.

I'm proposing to go further, by replacing or augmenting (a) and (b) with "rebuild all of stdenv". If this puts too much burden on Hydra it could run weekly or less often.

After spending the last two weeks digging around in the stdenv-bootstrap process I'm a little bit worried about the stdenv-bootstrap process making subtle undocumented assumptions about the bootstrap-files. We might not be able to document all these assumptions, but we can at least use automated testing to alert ourselves immediately at whatever commit they end up being violated.

Ah, I see. Rebuilding the stdenvs is relatively expensive, but I expect it wouldn't happen often, except on changes that already rebuild very many packages anyway.

Ah, I see. Rebuilding the stdenvs is relatively expensive, but I expect it wouldn't happen often, except on changes that already rebuild very many packages anyway.

It could be done only after merges from staging to master.

Anything that causes stdenv to rebuild ought to go through staging anyways.

Additional commit 3ecf7ee fixes an embarrassing oversight.

The force-push following this commit will collapse the PR to a single commit (no other changes).

OMG, this somehow caused removal of the jobs on Hydra and noone's noticed for months? https://hydra.nixos.org/eval/1758946#tabs-removed

Could it be due to job name clash between release.nix / all-packages.nix?

$ git grep -F 'stdenvBootstrapTools =' | cat
pkgs/top-level/all-packages.nix:  stdenvBootstrapTools = if stdenv.hostPlatform.isDarwin then
pkgs/top-level/release.nix:      stdenvBootstrapTools = with lib;

OMG, this somehow caused removal of the jobs on Hydra and noone's noticed for months?

Doesn't Hydra email the meta.maintainers of the removed packages when that happens?

Cross-compiled bootstrapTools seem unaffected.

The job name for cross-built bootstrap tools is bootstrapTools rather than stdenvBootstrapTools. This corroborates trofi's hypothesis.

OMG, this somehow caused removal of the jobs on Hydra and noone's noticed for months?

Doesn't Hydra email the meta.maintainers of the removed packages when that happens?

Hydra hasn't emailed in years, AFAIK.

OMG, this somehow caused removal of the jobs on Hydra and noone's noticed for months?

Doesn't Hydra email the meta.maintainers of the removed packages when that happens?

Hydra hasn't emailed in years, AFAIK.

Oh my. Why not? Being notified of build failures seems like such a useful feature.

Also is there any way to mass-download the underlying data which are used to render https://hydra.nixos.org/project/nixpkgs? I would really like to be able to query these data from a shell prompt and with lower latency.

Let's rename the attribute to make job names distinct on hydra: #182058